Image compounding for mixed transducer arrays

ABSTRACT

A method of imaging may include receiving a first signal from one or more array elements of a first type in a mixed transducer array, receiving a second signal from one or more array elements of a second type in the mixed transducer array where at least one of the first type and the second type is an optical sensor, generating a first image from the first signal and a second image from the second signal, and combining the first image and the second image to generate a compound image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. patent application Ser. No.63/104,886 filed on Oct. 23, 2020, which is incorporated herein in itsentirety by this reference.

TECHNICAL FIELD

The present disclosure generally relates to the field of imaging, and inparticular to methods and devices that enable forming a compound imagefrom images acquired by a mixed array including an array of opticalsensors and other transducers. The methods and devices disclosed hereininclude optical sensors that have high sensitivity and/or highoperational bandwidth for improved imaging performance.

BACKGROUND

Ultrasound sensing is used in various industries including medicalimaging and medical diagnosis due to a number of advantages. Forexample, ultrasound sensing utilizes ultrasound signal which has aremarkable penetration depth. Moreover, ultrasound imaging is known tobe an advantageously non-invasive form of imaging, as it is based onnon-ionizing radiation.

Various known ultrasound transducers used in ultrasound imaging havenumerous drawbacks. For example, some ultrasound transducers are made ofpiezoelectric material, such as lead zirconate titanate (PZT). However,the 6-dB bandwidth of PZT materials is generally limited to only about70%. Certain composite PZT materials have a slightly increasedbandwidth, but still only achieve a bandwidth of up to about 80%. Asanother example, single crystal materials have increasingly been used inan effort to improve performance of ultrasound probes but have lowerCurie temperatures and are brittle. Another type of transducer materialis silicon, which can be processed to build Capacitive MicromachinedUltrasound Transducer (CMUT) probes that can have increased bandwidth.However, CMUT probes are not very sensitive or reliable. Moreover, CMUTprobes have several operational limitations. For example, CMUT probesare nonlinear sensors and, therefore, are not generally suitable forharmonic imaging. Thus, there is a need for ultrasound probes with mixedtransducer arrays (mixed arrays) that include sensors with higherbandwidth and sensitivity. Moreover, there is a need for back enddevices, and/or front end devices to process signals and/or imagesgenerated by the mixed arrays.

SUMMARY

Generally, in some variations, an apparatus (e.g., an image compoundingsystem) for imaging (e.g., ultrasound imaging a patient) may include amixed transducer array including one or more array elements of a firsttype configured to receive a first signal, and one or more arrayelements of a second type configured to receive a second signal, whereinat least one of the first type and the second type is an optical sensor.The apparatus may further include one or more processors configured togenerate a first image from the first signal and a second image from thesecond signal, and combine the first image and the second image togenerate a compound image.

In some variations, the array elements of the first type may include anon-optical transducer and the array elements of the second type mayinclude an optical sensor. The one or more array elements of the firsttype may include, for example, a piezoelectric transducer, a singlecrystal material transducer, a piezoelectric micromachined ultrasoundtransducer (PMUT), or a capacitive micromachined ultrasonic transducer(CMUT). The optical sensor may include, for example, a whisperinggallery mode (WGM) optical resonator, a microbubble optical resonator, aphotonic integrated circuit (PIC) optical resonator, a microsphereresonator, a microtoroid resonator, a microring resonator, a microbottleresonator, a microcylinder resonator, and/or a microdi sk opticalresonator.

In some variations, the array elements of the second type may includeoptical sensors with different characteristics (e.g., different designand/or different operating parameters). For example, in some variations,the array elements of the second type may include one or more highquality factor (high Q) optical sensors, and one or more low quality(low Q) optical sensors. Additionally or alternatively, the arrayelements of the second type may include one or more tunable opticalresonators configured to operate as a high Q optical resonator, and/orthe array elements of the second type may include one or more tunableoptical resonators configured to operate as a low Q optical resonator.For example, such tunable optical resonators may be selectively operablein a high Q or low Q mode, depending on imaging settings, etc.

Furthermore, in some variations, the mixed transducer array may includea combination of one or more non-optical transducers and multiple typesof optical sensors. For example, the mixed transducer array may includeone or more array elements of a first type including at least onenon-optical transducer, one or more array elements of a second type mayinclude at least one type of optical sensor, and one or more arrayelements of a third type may include at least another type of opticalsensor. The one or more processors may be further configured to generatea third image from the third signal, and combine the first image, thesecond image, and the third image to generate a compound image.Different types of optical resonator sensors may include, for example, ahigh Q optical resonator and a low Q optical resonator (or a tunableoptical resonator sensor configured to operate as a high Q opticalresonator or a low Q optical resonator). As another example, differenttypes of optical resonator sensors may include a broad bandwidth opticalresonator and an ultra-sensitive optical resonator.

In some variations, one or more array elements of the mixed transducerarray (e.g., transducers) may transmit acoustic signals at a fundamentalfrequency f. In response, the one or more array elements of the firsttype, the second type, or both the first type and the second type mayproduce one or more responses upon receiving harmonic (includingsuper-harmonic and sub-harmonic) acoustic echoes corresponding to thetransmitted acoustic signal. The one or more array elements of thesecond type may have a bandwidth ranging from at least f/M to Nf, whereM and N are integers greater than 1. In some variations, the one or morearray elements of the first type may transmit acoustic signals at afirst fundamental frequency f₁ and a second fundamental frequency f₂. Inresponse, the one or more array elements of the second type may produceone or more optical responses upon receiving acoustic echoes thatcorrespond to a frequency of one or more linear combinations nf₁+mf₂,wherein n and m are integers such that nf₁+mf₂ is a positive number. Atleast one of the first image and the second image may be or include aharmonic image.

In some variations, the one or more processors may be configured tofilter the various signals from the different types of array elements inthe mixed transducer array, using one or more suitable filters. Suchsuitable filters may include, for example, a harmonic band-pass filterthat may enable extraction of the harmonic signals, includingsub-harmonic and super harmonic signals.

Combining the first image and the second image may be performed by asuitable compounding algorithm. For example, the one or more processorsmay be configured to combine the first and second images at least inpart by determining an average of the first image and the second image.For example, the one or more processors may be configured to combine thefirst and second images at least in part by determining an arithmetic orgeometric average of the first image and the second image. Additionallyor alternatively, the one or more processors may be configured tocombine the first and second images at least in part by determining aweighted average of the first image and the second image. In somevariations, such weighted averaging may include determining one or morecompounding coefficients for the first and second images, where thefirst and second images may be combined based on the one or morecompounding coefficients.

For example, in some variations, the one or more processors may beconfigured to determine one or more compounding coefficients at least inpart by transforming the first and second images to first and secondtransform domain images using at least one transformation operator,determining one or more transform domain compounding coefficients forthe first and second transform domain images, and inverse transformingthe one or more transform domain compounding coefficients to determinethe one or more compounding coefficients for the first and secondimages. The transform domain compounding coefficients may be determined,for example, at least in part by applying one or more coefficientcompounding rules (e.g., predetermined, heuristic-based, or learnedrules, etc.) to the first and second transform domain images. Thetransformation operator may include any suitable kind of transformationthat supports 1:1 forward and backward transformations (e.g., FourierTransform, a Discrete Wavelet Transform (DWT), a Discrete CosineTransform (DCT), or a Wave Atom Transform).

In some variations, the one or more processors may additionally oralternatively be configured to determine one or more compoundingcoefficients at least in part by determining a first image qualityfactor map for the first image and a second image quality factor map forthe second image, and determining a first compounding coefficient forthe first image based on the first image quality factor map, and asecond compounding coefficient for the second image based on the secondimage quality factor map.

Additionally or alternatively, in some variations, the one or moreprocessors may be configured to determine one or more compoundingcoefficients at least in part by determining a local entropy of eachpixel in the first image and a local entropy of each pixel in the secondimage, and determining one or more compounding coefficients based on thedetermined local entropies.

Other suitable techniques for determining compounding coefficientsinclude determining one or more compounding coefficients at least inpart by applying a linear filter (e.g., Difference of Gaussian filter)to each of the first and second images for estimating and weightingimage content, determining one or more compounding coefficients as afunction of imaging depth, and/or applying a saturation mask thatreduces weight (e.g., compounding coefficient) of at least a portion ofthe first image and/or second image that has exceeded a predeterminedsaturation threshold.

In other words, the one or more processors may be configured to combineimages from different types of sensors in the mixed transducer arrayusing one or more suitable compounding techniques as described herein,including, for example, one or more of arithmetic averaging, geometricaveraging, transform domain compounding, image quality factor-based(IQF) compounding, local entropy weighted compounding, image contentweighted compounding, depth dependent weighted compounding, orsaturation masking, etc.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary image compounding system witha mixed array.

FIG. 2 is a block diagram of an exemplary image compounding system witha mixed array.

FIG. 3 is a block diagram of an exemplary image compounding system witha mixed array.

FIG. 4 is a block diagram of an exemplary image compounding system witha mixed array.

FIG. 5 is a block diagram of an exemplary image compounding system witha mixed array.

FIG. 6 is a flowchart of an exemplary method of performing imagecompounding on images acquired by a mixed array.

FIG. 7 is a flowchart of an exemplary method of performing imagecompounding on images acquired by a mixed array.

FIG. 8 is a flowchart of an exemplary method of performing imagecompounding on images acquired by a mixed array.

FIG. 9 is a flowchart of an exemplary method of performing imagecompounding on images acquired by a mixed array.

FIG. 10 is a flowchart of an exemplary method of performing imagecompounding on images acquired by a mixed array.

FIGS. 11A-11E show exemplary signals generated by a mixed array andharmonic filtering of the signals.

FIG. 12 shows a method of performing image compounding on imagesacquired by a mixed array.

FIG. 13 shows a method of performing image compounding on imagesacquired by a mixed array.

DETAILED DESCRIPTION

Non-limiting examples of various aspects and variations of the inventionare described herein and illustrated in the accompanying drawings.

Described herein are methods and devices for compounding (e.g.,combining) images acquired using mixed arrays that include multipletypes of array elements. Mixed arrays described herein include one ormore array elements of a first type and one or more array elements of asecond type different from the first type. The one or more arrayelements of the first type may be used to form a first image, while theone or more array elements of the second type may be used to form asecond image. The first type may include non-optical transducer such asa piezoelectric transducer, a single crystal material transducer, apiezoelectric micromachined ultrasound transducer (PMUT), and/or acapacitive micromachined ultrasonic transducer (CMUT), etc. The secondtype may include an optical sensor, which may be an interference-basedoptical sensor such as an optical resonator (e.g., a whispering gallerymode (WGM) optical resonator or photonic integrated circuit (PIC)optical resonator) or an optical interferometer. The optical sensor mayhave any suitable shape. For example, the optical sensor may be amicrobubble resonator, a microsphere resonator, a microtoroid resonator,microring resonators, a microbottle resonator, a microcylinder resonatorand/or a microdisk optical resonator, etc. The optical sensors have highsensitivity and/or broad bandwidth in reception of ultrasound signalscompared to other types of ultrasound sensors.

Various suitable combinations of non-optical transducers and one or moretypes of optical sensors may be included in the mixed transducer array.For example, in some variations, the array elements of the first typemay include a non-optical transducer, and the array elements of thesecond type may include an optical sensor. The one or more arrayelements of the first type may include non-optical transducers(non-optical sub-array) for transmitting acoustic signals and/ordetecting acoustic echoes to form a first image. The one or more arrayelements of the second type (e.g., optical sensors in an opticalsub-array) may be used to detect acoustic echoes (e.g., full spectrum,baseband, subharmonic, super-harmonic, and/or differential harmonic)that can be used to form a second image. The second image that isgenerated by highly sensitive and/or broad bandwidth optical sensors maybe used independently or can be combined with the first image to form aneven further improved image. Because of the high sensitivity and broadbandwidth of optical resonators, the image produced by optical sensorsmay have improved spatial resolution, improved contrast resolution,improved penetration depth, improved signal-to-noise ratio (SNR),improved tissue harmonic imaging, and/or improved Doppler sensitivity.However, because the optical sub-array and the non-optical sub-arrayintrinsically have different characteristics, compounded images producedby combining images generated using signals produced by different typeof sensors may have more features, better image quality and provide amore complete understanding of the underlying imaging target.

Moreover, the optical sensors do not generate ultrasound waves andtherefore are used together in mixed arrays with other transducers(e.g., piezoelectric, CMUT, and/or the like) that do generate ultrasoundwaves. The mixed arrays can be arranged in various configurations andinclude sensor elements with various noise levels, amplitude responses,phase delays, frequency ranges, and/or the like. Consequently,conventional beamforming methods and devices that are generally used forprobes with one type of sensor are not optimal for probes that use mixedarrays of multiple types of sensors. The optical resonators describedherein may have ultra-high quality factor (10³, 10⁵, 10⁷, 10⁹ and/or thelike) and hence ultra-high sensitivity for ultrasound detection butsmaller dynamic range. Such ultra-high quality factor optical resonatorsmay be particularly suitable for ultra-deep imaging but could sufferfrom undesirable nonlinear distortion in near field. On the other hand,the optical resonators can be designed to have a lower quality factorand hence a lower sensitivity compared to the optical resonators withultra-high quality factor. Such lower quality factor optical resonatorsmay be particularly suitable for imaging in the near field without theundesirable nonlinear distortion. Moreover, the optical resonators maysupport many different resonant modes. Therefore, an operation mode ofthe optical resonators can be switched from a first operation mode to asecond operation mode, for example, by switching the wavelength of alaser source coupled to the optical resonators. In some variations, animage compounding system may operate the optical resonators in theultra-high quality factor operation mode at a first time and in the lowquality factor operation mode at a second time. In some variations, animage compounding system can operate a first set of the opticalresonators in ultra-high quality factor operation mode and a second setof the optical resonators in low quality factor operation mode. Inaddition, sub-arrays consisting of different types of optical resonatorscan be deployed in the same image compounding system used to producedifferent images showing different aspects of the target. Combiningimages produced by different optical resonators or by operating opticalresonators in different operation modes using compounding algorithmssuch as those described herein can produce or otherwise generate imageswith a better image quality than images produced or generated by asingle type of sensor.

Accordingly, in some variations, the array elements of the second typemay include optical resonator sensors with different characteristics(e.g., different design and/or different operating parameters). Forexample, in some variations, the array elements of the second type mayinclude one or more high quality factor (high Q) optical resonators, andone or more low quality (low Q) optical resonators. Additionally oralternatively, the array elements of the second type may include one ormore tunable optical resonators configured to operate as a high Qoptical resonator, and one or more tunable optical resonators configuredto operate as a low Q optical resonator. For example, such tunableoptical resonators may be selectively operable in a high Q or low Qmode, depending on imaging settings, etc. Additionally or alternatively,the array elements of the second type may include one or more opticalresonator sensors that are designed for wide bandwidth, and one or moreoptical resonator sensors that are designed for ultra-high sensitivity.

Furthermore, in some variations, the mixed transducer array may includea combination of one or more non-optical transducers and multiple typesof optical sensors. Thus, different kinds of input images (e.g., fromnon-optical transducers and/or from one or more different kinds ofoptical sensors) may be combined using image compounding systems andmethods such as those described herein, to obtain a compounded image ofbetter quality than any individual input image.

Image Compounding Systems

FIG. 1 is a block diagram of an exemplary image compounding system 100with a mixed array. The image compounding system 100 includes a probe125, an imaging system 160, and a display 170. The probe 125 may beoperatively coupled to the imaging system 160. The probe 125 may receiveand/or transmit a set of signals (e.g., electrical signals,electromagnetic signals, optical signals, etc.) from/to the imagingsystem 160. The probe 125 includes a mixed array 110 that may receiveand/or transmit a set of signals (e.g., acoustic signals, etc.) from/toa medium for use in forming an image. The imaging system 160 may includea front end 140 and a back end 150 that may collectively determinephysical parameters (e.g., timing, location, angle, intensity, and/orthe like) of signals transmitted to the probe (e.g., via one or moretransmit channels), and post-process signals received by the probe 125(e.g., via one or more receive channels) to form an image. The imagingsystem 160 may also be coupled to the display 170 to transmit a set ofsignals (e.g., electrical signals, electromagnetic signals, etc.) to thedisplay 170. For example, in some variations, the display 170 may beconfigured to display the image produced by the imaging system 160(e.g., in a graphical user interface (GUI)). Additionally oralternatively, the imaging system 160 may receive signals from thedisplay 170. For example, the display 170 may further include aninteractive interface (e.g., touch screen, keyboard, motion sensor,and/or the like) to receive commands from a user of the imagecompounding system 100, such as to control operation of the imagecompounding system 100.

As shown in FIG. 1 , the probe 125 may include a mixed array 110, amultiplexer 120, and an optical sensor cable 130. The mixed array 110may include one or more non-optical array elements (e.g., PZTtransducers, CMUT transducers, etc.) and one or more optical arrayelements (e.g., optical sensors such as WGM resonators). The non-opticaltransducers may be configured to transmit acoustic waves, and in somevariations may be configured to additionally receive and detect acousticechoes in response to transmitted acoustic waves. The optical sensorsmay be configured to receive and detect echo signals with highsensitivity and/or broad bandwidth response. In some variations themixed array may be similar to any of the mixed arrays described inInternational Patent App. No. PCT/US2021/033715, which is incorporatedherein in its entirety by this reference. In some variations, the mixedarray may be configured to perform harmonic imaging as described inInternational Patent App. No. PCT/US2021/039551, which is incorporatedherein in its entirety by this reference. In some variations, the probe125 may be configured to iteratively scan across a field of view byusing the mixed array 110. In some variations, signals from the mixedarrays may be combined through a synthetic aperture technique, such astechniques described in International Patent App. No. PCT/US2021/049226,which is incorporated herein in its entirety by this reference. Suchsignals may be used to generate images using the optical sensors and/orthe non-optical transducers, as described in further detail below.

The mixed array 110 may include an array of transducer elements and maybe configured for operation in a 1 dimensional (1D) configuration, a1.25 dimensional (1.25D) array configuration, a 1.5 dimensional (1.5D)array configuration, a 1.75 dimensional (1.75D) array configuration, ora 2 dimensional (2D) array configuration. Generally, dimensionality ofthe ultrasound sensor array relates to the range of elevation beam width(or elevation beam slice thickness) that is achievable when imaging withthe ultrasound sensor array, and how much control the system over thesensor array's elevation beam size, foci, and/or steering throughout animaging field (e.g., throughout imaging depth). A 1D array has only onerow of elements in elevation dimension and a fixed elevation aperturesize. A 1.25D array has multiple rows of elements in elevation dimensionand a variable elevation aperture size, but a fixed elevation focalpoint via an acoustic lens. A 1.5D array has multiple rows of elementsin elevation dimension, a variable elevation aperture size, and avariable elevation focus via electronic delay control. A 1.75D array isa 1.5D array with additional elevation beam steering capability. A 2Darray has large numbers of elements in both lateral and elevationdimensions to satisfy the minimum pitch requirement for large beamsteering angles in both the lateral and elevation directions.

In some variations, the image compounding system may be configured toturn a 1.5D array configuration or a 2D array configuration into a 1Darray configuration. The mixed array 110 may include a large number(e.g., 16, 32, 64, 128, 256, 1024, 4096, 8192, 16384, and/or the like)of elements. In some variations, the mixed array 110 may be arranged ina rectangular configuration and may include N×M elements, where N is thenumber of rows and M is the number of columns. In some variations, forexample, the mixed array 110 includes one or more array elements of afirst type and one or more array elements of a second type, where thefirst type may be a piezoelectric transducer or other non-opticaltransducer configured to transmit ultrasound waves and the second typemay be an optical sensor such as an optical resonator. Non-opticaltransducers and optical sensors may be collectively positioned in arectangular arrangement, a curved arrangement, a circular arrangement,or a sparse array arrangement.

The non-optical transducer(s) in the mixed array 110 may include, forexample, a lead zirconate titanate (PZT) transducer(s), a polymer thickfilm (PTF) sensor(s), a polyvinylidene fluoride (PVDF) sensor(s), acapacitive micromachined ultrasound transducer (CMUT)(s), apiezoelectric micromachined ultrasound transducer (PMUT) (s), atransducer(s) based on single crystal materials (e.g., LiNbO₃(LN),Pb(Mg_(1/3)Nb_(2/3))—PbTiO₃ (PMN—PT), andPb(IninNb_(1/2))—Pb(Mg₁₃Nb_(2/3))—PbTiO₃ (PIN—PMN—PT)), and/or anytransducer suitable for acoustic sensing.

The optical sensor may be or include, for example, an interference-basedoptical sensor such as an optical interferometer or optical resonator(e.g., whispering gallery mode (WGM) optical resonator). In variationsin which the optical sensor is an optical resonator, the optical sensormay have any suitable shape or form (e.g., a microring resonator, amicrosphere resonator, a microtoroid resonator, a microbubble resonator,a fiber-based resonator, an integrated photonic resonator, a micro-diskresonator, and/or the like). In some variations, the optical sensors maybe/include, for example, Fabry-Perot (FP) resonators, fiber-basedresonators (e.g., fiber ring resonators), photonics crystal resonators,waveguide resonators, or any other suitable optical resonator that maylocalize optical energy in space and time. For example, in somevariations an optical resonator may be similar to any of the opticalresonators described in International Patent App. Nos. PCT/US2020/064094and PCT/US2021/022412, each of which is incorporated herein in itsentirety by this reference.

The optical resonators may include a closed loop of a transparent medium(e.g., glass, transparent polymer, silicon nitride, titanium dioxide, orany other material that is suitably optically transparent at anoperation wavelength of the optical resonator) that allows somepermitted frequencies of light to continuously propagate inside theclosed loop, and to store optical energy of the permitted frequencies oflight in the closed loop. The aforementioned is equivalent to say thatthe optical resonators may permit a propagation of modes (e.g.,whispering gallery modes (WGMs)) traveling the surface of the opticalresonators and corresponding to the permitted frequencies to circulatethe circumference of the resonator. Each mode corresponds to propagationof at least one frequency of light from the permitted frequencies oflight. The permitted frequencies of light and the quality factor of theoptical resonators described herein may be based at least in part ongeometrical parameters of the optical resonator, refractive index of thetransparent medium, and refractive indices of an environment surroundingthe optical resonator.

An optical resonator as described herein may have a set of resonantfrequencies including a first subset of resonator frequencies and asecond subset of resonant frequencies. In some variation, the opticalresonator may be operated at the first subset of resonant frequencieswith high quality factors. Alternatively or in addition, in somevariations, the optical resonator may be operated at the second subsetof resonant frequencies with low quality factors. The high qualityfactor subset of resonant frequencies may be suitable for operating athighly sensitive sensing probes (or sub-arrays) while the low qualityfactor subset of resonant frequencies may be suitable for high dynamicrange applications.

In some variations, the sensitivity of the optical resonator may becontrolled by tuning geometrical and/or characteristic materialparameters of the optical resonator for tunability of the quality factorof the optical resonator. In some variations, the space inside and/oraround the optical resonators may be filled with an ultrasonicenhancement material, such as for example, polyvinylidene fluoride,parylene, polystyrene, and/or the like. The ultrasonic enhancementmaterial may increase sensitivity of the optical resonators.

The optical resonators may be coupled to other components toreceive/transmit light. In some implementations, the opticalresonator(s) may be operatively coupled, via an optical medium (e.g.,optical fiber, a tapered optical fiber, free space medium, and/or thelike), to a light source (e.g., a laser, a tunable laser, an erbiumdoped fiber amplifier, and/or the like) and/or a photodetector (e.g., ap-doped/intrinsic/n-doped (PIN) diode). Acousto-optic systems based onoptical resonators may directly measure ultrasonic waves through thephoto-elastic effect and/or physical deformation of the resonator(s) inresponse to the ultrasonic waves (e.g., ultrasonic echoes). Therefore,the optical resonators may be considered as optoacoustic transducersthat may convert mechanical energy (e.g., acoustic energy) to opticalenergy. For example, in the presence of ultrasonic (or any pressure)waves, the modes traveling in a resonator may undergo a spectral shiftor amplitude change caused by changes in the refractive index and/orshape of the resonator. The spectral change may be easily monitored andanalyzed in the spectral domain using the photodetector. The amplitudechange may also be detected by the photodetector. The photodetectoreventually converts the optical energy (i.e., optical signal)propagating in the optical resonators and the optical fiber intoelectrical energy (i.e. electrical signal) suitable for processing withelectronic circuitry. Additional spatial and other information mayfurthermore be derived by monitoring and analyzing optical response ofoptical resonators among mixed arrays. Exemplary mixed transducer arraysare described herein. Additionally or alternatively, signals from theoptical resonator(s) can be processed by optical circuitry before beingconverted to electrical energy by photodetector(s).

The mixed array 110 may have the one or more non-optical array elements(e.g., ultrasound transducer or other non-optical transducer) and theone or more optical array elements (e.g., optical sensor such as a WGMoptical resonator) arranged in various configurations (similar to any ofthe mixed arrays described in International Patent App. No.PCT/US2021/033715, which was incorporated above). For example, in someconfigurations, the non-optical and optical array elements may becollectively positioned in a rectangular array including a number ofrows and a number of columns. The rectangular array may include N×Msensor elements, where N is the number of rows and M is the number ofcolumns and are both integers. In some implementations such as for a 2Darray, the number of rows and/or the number of columns may be greaterthan 31 rows and/or 31 columns. For example, a 2D mixed array mayinclude 64×96=6,144 sensor elements.

In some variations, mixed array 110 may include optical sensors ofmultiple different types. For example, as further described below,different types of optical sensors may include a broad bandwidth opticalresonator and an ultra-sensitive optical resonator. As another example,the mixed array 110 may include one or more high quality factor (high Q)optical resonators, and one or more low quality (low Q) opticalresonators. Additionally or alternatively, mixed array 110 may includeone or more tunable optical resonators configured to operate indifferent quality factor modes. For example, the tunable opticalresonators can be operated at a low quality factor (low Q) operationmode for a high dynamic response or a high quality factor (high Q)operation mode for a sensitive response. In some implementations, thetunable optical resonators may be or include a first set of tunableoptical resonators and a second set of tunable optical resonators thatmay be operated at different operation modes. In some implementations,the tunable optical resonators may be operated at the high Q operationmode at a first time interval and operated at the low Q operation modeat a second time interval. In other words, in some variations the mixedarray 110 may include one or more tunable optical resonators configuredto operate as a high Q optical resonator, and/or one or more tunableoptical resonators configured to operate as a low Q optical resonator.For example, such tunable optical resonators may be selectively operablein a high Q or low Q mode, depending on imaging settings, etc.

In some configurations, a spatial distribution of positions of multiplearray element types may be random. By using the sparse spatialdistribution of array elements, generation of grating lobes in an imageproduced by the mixed array may be reduced and/or prevented. A spatialdistribution of the array elements of a first type may be the same,similar to, or different from, a spatial distribution of the arrayelements of a second type. In some configurations, a spatialdistribution of positions of the array elements of a first type and asecond type may follow a dispositioning pattern (e.g., be the same,shift to the right by one cell among sensor elements, shift to down bytwo cells among sensor elements). In some instances, the one or morearray elements of a second type may be smaller than or the same as theone or more array elements of a first type.

The non-optical transducers in the mixed array 110 may be operativelycoupled to the multiplexer 120 that handles transmitted and/or receivedelectrical signals between the imaging system 160 and the non-opticaltransducers. The optical sensors in the mixed array 110 may beoperatively coupled to the optical sensor cable 130 that handlestransmitted and/or received optical signals between the imaging system160 and the optical sensors.

The multiplexer 120 functions to selectively connect individual systemchannels to desired array elements. The multiplexer 120 may includeanalog switches. The analog switches may include a large number of highvoltage analog switches. Each analog switch may be connected to anindividual system channel. As a result, the multiplexer 120 mayselectively connect an individual system channel from a set of systemchannels of the imaging system 160 to a desired transducer element ofthe mixed array 110.

The optical sensor cable 130 may include a dedicated optical path fortransmitting and/or receiving optical signals to and/or from the opticalsensors. The optical sensor cable 130 may include one or more opticalwaveguides such as, for example, fiber optical cable(s). Characteristicsof the optical sensor cable 130 may depend upon type of the opticalsignals, type of optical sensors, and/or an arrangement of opticalsensors. In some configurations, multiple optical sensors (e.g., theentire sub-array of the optical sensors, or any two or more opticalsensors forming a portion thereof) may be optically coupled to a singleoptical waveguide. Accordingly, signals from multiple optical sensorsmay be coupled into and communicated by a single optical waveguide. Insome configurations, the sub-array of the optical sensors may beoptically coupled to an array of optical waveguides in a 1:1 ratio(e.g., each optical sensor may be coupled to a respective opticalwaveguide). Accordingly, optical signals from the sub-array of theoptical sensors may be coupled to and communicated by one or moreoptical waveguides in the optical sensor cable 130 to the imaging system160.

The imaging system 160 may include a front end 140 and a back end 150.Generally, the front end 140 interfaces with the probe 125 to generateacoustic beams and receive electrical and/or optical signals. Forexample, the front end 140 may drive non-optical transducers (e.g.,transducers) in the probe to transmit ultrasound signals in predefinedbeam patterns, and may receive the reflected ultrasound signals from thenon-optical transducers and optical sensors in the mixed array in theprobe. The front end may also be tasked to perform both transmit andreceive beamforming. The back end 150 may include one or more processorsto process signals received from the mixed array 110 via the front endto generate images, a memory operatively coupled to the processor tostore the images, and/or a communication interface to present the imagesto a user (e.g., via graphical user interface). For example, the backend 150 may receive separately reconstructed images from the receivebeamformer in the front end, perform additional back end processes, andconduct image compounding operations. Various back end processes may beinvolved in the image formation, including digital signal processing(DSP), digital scan conversion (DSC), envelope detection, and/or thelike. To implement image compounding using optical sensors, the imagecompounding system may include specific implementations of a back endprocess for storing, analyzing, combining, and transmitting data,signals, and/or images. Such specific implementations are shown anddescribed below with respect to FIGS. 2-5 .

The display 170 may display a set of images generated by the imagingsystem 160. In some variations, the display 170 may additionally oralternatively include an interactive user interface (e.g., a touchscreen) and be configured to transmit a set of commands (e.g., pause,resume, and/or the like) to the imaging system 160. In some variations,the image compounding system 100 may further include a set of one ormore ancillary devices (not shown) used to input information to theimage compounding system 100 or output information from the imagecompounding system 100. The set of ancillary devices may include, forexample, a keyboard(s), a mouse(s), a monitor(s), a webcam(s), amicrophone(s), a touch screen(s), a printer(s), a scanner(s), a virtualreality (VR) head-mounted display(s), a joystick(s), a biometricreader(s), and/or the like (not shown).

FIG. 2 shows a block diagram of an exemplary image compounding system102 with a mixed array 110. As shown, the mixed array 110 may include anon-optical sub-array 113 and an optical resonator sub-array 114. Thefront end 140 may include a transmitter 142, a non-optical receiver 143,an optical resonator receiver 144, a transmit beamformer 145, anon-optical receive beamformer 146, and an optical resonator receivebeamformer 147. The back end 150 may include non-optical back endprocessor(s) 151 and optical resonator back end processor(s) 152. Thenon-optical back end processor(s) 151 and optical resonator back endprocessor(s) 152 may involve performing including digital signalprocessing (DSP), digital scan conversion (DSC), envelope detection,and/or the like.

The transmit beamformer 145 generates various transmit waveforms basedon transmit beamformer settings 181. The waveforms may be amplified bythe transmitter 142 that may include analog circuitry, digitalcircuitry, and/or computer systems, before being applied to thenon-optical sub-array 113. After receiving the waveforms and/oramplified waveforms by the transmitter 142 the non-optical sub-array 113may generate a set of acoustic waves (e.g., ultrasound signals) toward atarget. The acoustic waves insonify the target, which in turn reflectspart of the acoustic waves (i.e., echo signals) back to the mixed arrayprobe. The non-optical receiver 143 receives the echo signals detectedby the non-optical transducers and processes them to produce digitizedsignals as the output. The signals detected by the optical resonatorsub-array 114 may be processed and digitized by the optical resonatorreceiver 144. The non-optical resonator receive beamformer 146, theoptical receive beamformer 147, the non-optical back end processors 151,and the optical back end processors 152, use the signals processed bythe two receivers to form non-optical images 182 and optical resonatorimages 183. The non-optical images 182 and optical resonator images 183often have different characteristics. The different characteristics ofnon-optical images 182 and optical resonator images 183 may depend onfactors including an arrangement of sensing elements (non-opticaltransducer or optical resonator) in the mixed array, physical parametersof the sensing elements, and/or the like.

FIG. 3 shows a block diagram of an exemplary image compounding system103 with a mixed array 110 that includes optical resonator sensorsincluding sub-arrays with different quality factors (Q factors). Asshown, the mixed array 110 may include a non-optical sub-array 113, ahigh quality factor (high Q) optical resonator sub-array 115, and a lowquality factor (low Q) optical resonator sub-array 116. The front end140 may include a transmit beamformer 145, a transmitter 142, a high Qoptical resonator receiver 148 that receives signals from the high Qoptical resonator sub-array, a low Q optical resonator receiver 149 thatreceives signals from the low Q optical resonator sub-array, and anoptical resonator receive beamformer 147. Although separate opticalresonator receivers (high Q optical resonator receiver 148 and low Qoptical resonator receiver 149) are shown in FIG. 3 as receiving signalsfrom high Q optical resonators and low Q optical resonators,respectively, it should be understood that in some variations, thereceivers 148 and 149 may be replaced by one or more receivers that mayreceive a wide range of Q factor signals. For example, a single receivermay dynamically be tuned or otherwise configured to receive low Qsignals (e.g., in one or more “low Q” modes) and tuned or otherwiseconfigured to receive high Q signals (e.g., in one or more “high Q”modes). The single receiver may be dynamically configured across aspectrum of Q factors, or may be operable among different discrete modescorresponding to respective ranges of Q factors. The back end 150 mayinclude one or more optical resonator back end processors 152. Theoptical resonator back end processors 152 may involve performing one ormore techniques including digital signal processing (DSP), digital scanconversion (DSC), envelope detection, and/or the like.

Signals acquired by the high Q optical resonator sub-array 115 maygenerate one or more high sensitivity images 184, where features withlower reflectivity or weaker signals from deep depth may be bettervisualized and features with high reflectivity or strong signals fromshallow depth may be saturated. On the other hand, the low Q opticalresonator sub-array generates one or more high dynamic range images 185that may miss smaller and lower reflective features or weaker signalsfrom deep depth. The one or more high sensitivity images 184 and the oneor more high dynamic range images 185 may be used in the opticalresonator back end processor(s) 152 to generate a compound image thatincludes the advantages of signals of each of the high Q and low Qoptical resonator sub-arrays.

As shown in FIG. 3 , in some variations, the high Q optical resonatorsub-array 115 and the low Q optical resonator sub-array 116 may sharethe optical resonator receive beamformer 147 and the optical resonatorback end processor(s) 152. Alternatively, in some variations, the high Qoptical resonator sub-array 115 and the low Q optical resonatorsub-array 116 may have different respective receive beamformers and/ordifferent respective back end processor(s). For example, the high Qoptical resonator sub-array 115 may be operatively coupled to a high Qoptical resonator receive beamformer (not shown) and a high Q opticalresonator back end process (not shown), and the low Q optical resonatorsub-array 116 may be operatively coupled to a low Q optical resonatorreceive beamformer (not shown) and a low Q optical resonator back endprocess (not shown).

In some variations, the front end 140 may further include a non-opticalreceiver and a non-optical receive beamformer (e.g., non-opticalreceiver 143 and non-optical receive beamformer 146 as shown anddescribed with respect to FIG. 2 ). Consequently, the back end 150 mayalso include non-optical back end processor(s) such as non-optical backend processor(s) 151 that produce non-optical images 182 as shown anddescribed with respect FIG. 2 . Therefore, the image compounding system103 may be configured to form a compound image based on high sensitivityimages 184 and high dynamic range images 185, and optionallyadditionally based on non-optical images 182.

FIG. 4 shows a block diagram of an exemplary image compounding system104 with a mixed array 110 that is similar to the image compoundingsystem 103 shown and described above with respect to FIG. 3 , exceptthat the mixed array 110 includes a tunable optical resonator sub-array117 that is operable in two or more modes with different Q factorvalues. Tuning for different modes may be accomplished by, for example,selectively modifying ambient temperature around the mixed array 110,and/or changing the optical wavelength. Such a tunable optical resonatorsub-array 117 may be used to acquire both high sensitivity images andhigh dynamic range images. For example, in some variations, at least oneoptical resonator in the tunable optical resonator sub-array 117 mayreceive signals at multiple times in response to different sets oftransmission sequences, where the at least one optical resonatoroperates in a high Q mode at one time, and in a low Q mode at adifferent time. In other words, in some variations, at least a portionof the tunable optical resonator sub-array 117 may be operated at afirst time interval and a second time interval not overlapping the firsttime interval, where at least a portion of the tunable optical resonatorsub-array 117 may be operated as a high Q optical resonator at the firsttime interval to generate the high sensitivity images 184, and as a lowQ optical resonator at the second time interval to generate the highdynamic range images 185. In some variations, at least one tunableoptical resonator may operate in a high Q mode before operating in a lowQ mode. Additionally or alternatively, at least one tunable opticalresonator may operate in a low Q mode before operating in a high Q mode.At least two sets of transmission sequences may be performed to insonifythe target multiple times to acquire signals from both the high Qoptical resonator receiver 148 and the low Q optical resonator receiver.

Additionally or alternatively, in some variations, at least a firstportion (e.g., a first set) of the tunable optical resonator sub-array117 may be consistently designated to operate in a high Q mode, and atleast a second portion (e.g., a second set) of the tunable opticalresonator sub-array 117 may be consistently designated to operate in alow Q mode. Signals from the first portion of the tunable opticalresonators may be received by the high Q optical resonator receiver 148,and signals from the second portion of the tunable optical resonatorsmay be received by the low Q optical resonator receiver 149. In somevariations in which the tunable optical resonator sub-arraysimultaneously includes some optical resonators tuned to operate in ahigh Q mode and some optical resonators tuned to operate in a low Qmode, the mixed array 104 may be functionally similar to the mixed array103 shown and described above with respect to FIG. 3 . Similar to thatdescribed above with respect to FIG. 3 , although separate opticalresonator receivers (high Q optical resonator receiver 148 and low Qoptical resonator receiver 149) are shown in FIG. 4 as receiving high Qsignals and low Q signals, respectively, it should be understood that insome variations, the receivers 148 and 149 may be replaced by one ormore receivers that may receive a wide range of Q factor signals. Forexample, a single receiver may dynamically be tuned or otherwiseconfigured to receive low Q signals (e.g., in one or more “low Q” modes)and tuned or otherwise configured to receive high Q signals (e.g., inone or more “high Q” modes). The single receiver may be dynamicallyconfigured across a spectrum of Q factors, or may be operable amongdifferent discrete modes corresponding to respective ranges of Qfactors.

As shown in FIG. 4 , the mixed array 110 may include a non-opticalsub-array 113 and a tunable optical resonator sub-array. The front end140 may include a transmit beamformer 145, a transmitter 142, a high Qoptical resonator receiver 148, a low Q optical resonator receiver 149,and an optical resonator receive beamformer 147. The non-opticalsub-array 113 in the mixed array 110 may transmit a set of acousticsignals, and the tunable optical resonators sub-array may receive a setof acoustic echoes in response to the acoustic signals. The tunableoptical resonator sub-array 117 may be operatively coupled to aphotodetector configured to generate a first signal and a second signal,where the first signal includes a readout from at least a portion of thetunable optical resonator sub-array 117 operating in a high Q mode, andthe second signal includes a readout from at least a portion of thetunable optical resonator sub-array 117 operating in a low Q mode. Thehigh Q optical resonator receiver 148 and the low Q optical resonatorreceiver 149 may receive the first signal and the second signal,respectively. The back end 150 may include an optical resonator back endprocessor(s) 152. The optical resonator back end processor(s) 152 mayperform operations including digital signal processing (DSP), digitalscan conversion (DSC), envelope detection, and/or the like on the firstsignal and the second signal to generate high sensitivity images 184 andhigh dynamic range images 185. The back end 150 may be furtherconfigured to combine the high sensitivity images 184 and the highdynamic range images 185 to generate a compound image that includes theadvantages of signals of each of the high Q and low Q modes of thetunable optical resonator sub-array 117

In some variations, multiple transmission sequences are transmittedusing the transmit beamformer settings 181, the transmit beamformer 145,the transmitter 142, and the non-optical sub-array 113 to insonify atarget multiple times. For example, the non-optical sub-array 113 maytransmit a first transmission sequence and a second transmissionsequence. In response, the tunable optical resonator sub-array 117 mayacquire the first signal in response to the first transmission sequenceand the second signal in response to the second transmission sequence.The back end may then produce the first image from the first signal andproduce the second image from the second signal.

FIG. 5 shows a block diagram of an exemplary image compounding system105 with a mixed array 110 that includes optical resonators in both asub-array with broad bandwidth and a sub-array with high sensitivity.For example, the mixed array may include a non-optical sub-array 113, abroad bandwidth optical resonator sub-array 118 and an ultra-sensitiveoptical resonator sub-array 119. The broad bandwidth optical resonatorsub-array 118 may capture signal outside of the baseband of thetransmitted acoustic waves, such as super-harmonics and subharmonicsfrom tissue and/or contrast agents (e.g., as described in InternationalPatent App. No. PCT/US2021/039551, which was incorporated above byreference). The ultra-sensitive optical resonator sub-array 119 maycapture signals from deeper regions in and out of the baseband.

The non-optical sub-array 113 may be operatively coupled to thetransmitter 142, which is operatively coupled to the transmit beamformer145 receiving transmit beamformer settings 181. The non-opticalsub-array 113 transmits acoustic signals towards a target and receivesacoustic echoes in response to the acoustic signals. The non-opticalsub-array 113 may be additionally operatively coupled to the non-opticalreceiver 143 and the non-optical receive beamformer 146 in the front end140 to generate a first signal in response to the acoustic echoesreceived at the non-optical sub-array 113. The non-optical back endprocessor(s) 151 may analyze the first signal to generate a first image(non-optical image(s) 182) that visualizes the target with conventionalspatial resolution and imaging depth. The broad bandwidth opticalresonator sub-array 118 and the ultra-sensitive optical resonatorsub-array 119 may be operatively coupled to the optical resonatorreceiver 144 and optical resonator receive beamformer 147. The opticalresonator back end processor(s) 152 may be used to process signals fromthe two optical resonator sub-arrays 118 and 119 to produce one or moreimages (e.g., fundamental frequency images, super-harmonic images,sub-harmonic images, etc.) and one or more high sensitivity images. Forexample, a second signal originating from the broad bandwidth opticalresonator sub-array 118 may be used to generate a second image (harmonicimage(s) 186), and/or a third signal originating from theultra-sensitive optical resonator sub-array 119 may be used to generatea third image (high sensitivity image(s) 184). Therefore, the imagecompounding system 105 may achieve enhanced spatial resolution andimaging depth at the same time.

After the first image(s), the second image(s), and/or the third image(s)are separately generated using the first signal, the second signal,and/or the third signal from the non-optical sub-array 113, the broadbandwidth optical resonator sub-array 118, and the ultra-sensitiveoptical resonator sub-array 119, respectively, an image compoundingalgorithm may be used to combine the first image, the second image,and/or the third image and produce a compound image as further describedbelow.

Methods of Performing Image Compounding

FIGS. 6-10 described below illustrate aspects of exemplary methods ofperforming image compounding based on images received from a mixed arraydescribed above. Although the methods are primarily described withreference to optical resonator sensors, it should be understood thatthey may similarly be performed using signals from optical sensors ofother kinds (e.g., optical interferometer). The methods of performingimage compounding may be executed by an image compounding computingdevice that is part of (e.g., back end 150 as shown and described withrespect to FIGS. 1-5 ) and/or is operatively coupled to an imagecompounding system (such as the image compounding system 100 shown anddescribed with respect to FIG. 1 ). The image compounding computingdevice may include a set of electronic circuitries such as a processor,a memory, and a communication interface. The processor may include, forexample, a hardware based integrated circuit (IC) or any other suitabledevice to run or execute a set of instructions/codes. For example, theprocessor may include a general purpose processor, a central processingunit (CPU), an accelerated processing unit (APU), an applicationspecific integrated circuit (ASIC), a microprocessor, a fieldprogrammable gate array (FPGA) chip, a graphics processing unit (GPU), adigital signal processing (DSP) chip, and/or the like. The memory maystore, for example, code that includes instructions to cause theprocessor to perform one or more processes or functions (e.g., filteringsignals, amplifying signals, phase matching, noise reduction, selectingapertures, and/or the like). The memory may be/include, for example, amemory buffer, a random access memory (RAM), a read-only memory (ROM), aflash drive, a secure digital (SD) memory card, and/or the like. Thecommunication interface may be/include a universal serial bus (USB)interface, a peripheral component interconnect express (PCIe) interface,or a hardware component that is operatively coupled to the processorand/or the memory and may enable communication of the image compoundingcomputing device with components of the image compounding system and/orin some variation, external device and/or network of devices (e.g., theInternet).

The image compounding computing device may include an application as asoftware stored in the memory and executed by the processor. Forexample, the application may include code to cause the processor toselect aperture, analyze signals, generate an image, and/or the like.Alternatively, the application may be implemented on a hardware-baseddevice. For example, the application may include a digital circuit(s) oran analog circuit(s) that may cause the image compounding computingdevice to filter signals, amplify signals, and/or delay signals.

FIG. 6 is a flowchart of an exemplary method 600 of performing imagecompounding on images acquired by a mixed array. In someimplementations, the method may be performed with the compound imagingsystem 102 (e.g., back end 150) as shown and described with respect toFIG. 2 . The method 600 may include initiating image acquisition (601)(e.g., upon receipt of an indication to begin acquisition). The method600 may further include transmitting a non-optical signal (602) followedby receiving a non-optical signal (603) and receiving an opticalresonator signal (604) (or other optical sensor signal). The method mayiterate 602, 603, and/or 604 until all transmit steps desired totransmit acoustic signals from all non-optical array elements and allreceive steps to receive acoustic echoes from all non-optical arrayelements and optical array elements of the mixed array 110 are executed.Once all desired transmission and receiving have been performed for atleast one desired compound image (605), the method 600 may furtherinclude generating or forming non-optical images (606) and generating orforming optical resonator images (607) using the front end 140 and backend 150 of the compound imaging system 102. The back end 150 may thenapply image domain filters to the non-optical images and opticalresonator images (608, 609). The image domain filters may bespecifically designed according to the image characteristics of eachtype of images. The method 600 may include combining (e.g., using acompounding algorithm such as those described below) the non-opticalimages and optical resonator images (610) and producing the compoundimages (611). Generally, in some variations, the compound images may,for example, be formed utilizing dynamically-determined weight maskswith compounding coefficients that indicate which features of thenon-optical images and which features of the optical resonator imagesmay be included in each compound image.

Additionally or alternatively, in some variations, compound images maybe formed utilizing static weight masks that may be pre-determined andstored for use during later image compounding processes. For example, ifan image compounding method is not dependent on the content of theimages (such as method 700) or is static, weight masks may bepre-computed and stored in a memory of the image compounding system.Executing image compounding methods based on pre-computed weight masksmay be processed faster and more efficiently by a processor of the imagecompounding system. FIG. 7 is a flowchart of an exemplary method 700 ofperforming image compounding on images acquired by a mixed array, whereimage compounding utilizes pre-computed weight masks with compoundingcoefficients.

The method 700 may include steps 601-607 as shown and described withrespect to FIG. 6 . However, the method 700 may further includeretrieving pre-computed weight masks (708). The method 700 may thenperform weighted average of non-optical images and optical resonatorimages to generate combined images (709). The weighted average mayinclude arithmetic averaging, geometric averaging, depth-dependentweighting, region-based weighting, and/or the like. The method 700 mayfurther include filtering the combined images (710) and producingcompound images (711).

FIG. 8 is a flowchart of an exemplary method 800 of performing imagecompounding on images acquired by a mixed array. In someimplementations, the method 800 may be performed with the compoundimaging system 103 as shown and described with respect to FIG. 3 . Themethod 800 may include initiating image acquisition (801) (e.g., uponreceipt of an indication to begin acquisition). The method 800 mayfurther include transmitting a non-optical signal (802) followed byreceiving a high quality factor (high Q) optical resonator and/or lowquality factor (low Q) optical resonator signal (803). The method 800may iterate 802 and 803 until all transmit steps desired to transmitacoustic signals from all non-optical array elements and all receivesteps to receive acoustic echoes from all high Q optical resonator arrayelements and low Q optical resonator array elements are executed. Onceall desired transmitting and receiving have been performed for at leastone desired compound image (804), the method 800 may further includegenerating or forming high Q optical resonator images (805) (alsoreferred to as high sensitivity images) and generating or forming low Qoptical resonator images (806) (also referred to as high dynamic rangeimages) using the front end 140 and back end 150 of the compound imagingsystem 103. The back end 150 may then filter the high Q opticalresonator images (807) and filter the low Q optical resonator images(808). The method 800 may include combining the high Q optical resonatorimages and the low Q optical resonator images (809) (e.g., using acompounding algorithm) and producing the compound images (810). Similarto method 700, in some variations (e.g., if method 800 is not dependenton the content of the images or is static), weight masks may bepre-computed and stored in a memory of the image compounding system 103for faster processing.

FIG. 9 is a flowchart of an exemplary method of performing imagecompounding on images acquired by a mixed array. In someimplementations, the method 900 may be performed with the compoundimaging system 104 as shown and described with respect to FIG. 4 . Themethod 900 may include initiating image acquisition (901) (e.g., uponreceipt of an indication to begin acquisition). The method 900 mayfurther include transmitting a non-optical signal (902) followed byreceiving an optical resonator signal from at least one tunable opticalresonator signal operating in a high Q mode (903). In some instances,the optical resonators may be operated at the high Q setting by choosingthe optical wavelength (of a light source) to match a resonancefrequency in which the quality factor of the resonance is high. Themethod 900 may further include transmitting a non-optical signal (904)followed by receiving an optical resonator signal from at least onetunable optical resonator signal operating in a low Q mode (905). Whilethe flowchart FIG. 9 depicts receiving signals from optical resonatorsin high Q mode prior to receiving signals from optical resonators in lowQ mode, it should be understood that alternatively, signals from opticalresonators in low Q mode may be received prior to receiving signals fromoptical resonators in high Q mode. The method 900 may iterate the902-905 until all transmit steps desired to transmit acoustic signalsfrom all non-optical array elements and all receive steps to receiveacoustic echoes at all Q optical resonator array elements in low Qsetting and high Q setting are executed. Once all desired transmittingand receiving has been performed for at least one desired compound image(906), the method 900 may further include generating or forming high Qoptical resonator images (907) and generating or forming low Q opticalresonator images (908) using the front end 140 and back end 150 of thecompound imaging system 104. The back end 150 may then filter the high Qoptical resonator images (909) and filter the low Q optical resonatorimages (910). The method 900 may include combining the high Q opticalresonator images and the low Q optical resonator images (911) (e.g.,using a compounding algorithm) to produce the compound images (912).Similar to methods 700 and 800, in some variations (e.g., if method 900is static), weight masks may be pre-computed and stored in a memory ofthe image compounding system 104 for faster processing.

FIG. 10 is a flowchart of an exemplary method of performing imagecompounding on images acquired by a mixed array. In someimplementations, the method 1000 may be performed with the compoundimaging system 105 as shown and described with respect to FIG. 5 . Themethod 1000 may include initiating image acquisition (1001) (e.g., uponreceipt of an indication to begin acquisition). The method 1000 mayfurther include transmitting a non-optical signal (1002) followed byreceiving a non-optical signal (1003) and receiving an optical resonatorsignal (1004) (e.g., from a broad bandwidth optical resonator sub-arrayand/or an ultra-sensitive optical resonator sub-array). The method 1000may iterate 1002-1004 until all transmit steps desired to transmitacoustic signals from all non-optical array elements and all receivesteps to receive acoustic echoes at all non-optical array elements andoptical resonator array elements are executed. Once all desiredtransmitting and receiving steps for at least one desired compound imageare performed (1005), the method 1000 may further include generating orforming non-optical images (1006), generating or forming a harmonicoptical resonator images (1007), and generating or forming highsensitivity optical resonator images (1008) using the front end 140 andback end 150 of the compound imaging system 105. The back end 150 maythen filter the non-optical images (1009), filter the harmonic opticalresonator images (1010), and filter the high sensitivity opticalresonator images (1011). The filtering of the harmonic optical resonatorimages (i.e. low Q optical resonator images) may include executing a setof band pass filters and/or a set of one dimensional signal filters toextract the components in the sub-harmonics and/or super-harmonicsbands. Subsequently, these filtered signals are used to form harmonicsimages at each of the selected bands. The method 1000 may includecombining the non-optical images, the harmonic optical resonator images,and the high sensitivity optical resonator images 1012 (e.g., using acompounding algorithm) to produce the compound images (1013).

As described above, in forming the harmonic optical resonator images,the optical resonator signals may be processed with a filter bankcomprising one or more filters. FIGS. 11A-11E show exemplary signalsgenerated by a mixed array and harmonic filtering of the signals. Asshown in FIG. 11A, a first signal 1101 is received by a wide-bandoptical resonator. By executing a transformation such as, for example, aFourier Transform the first signal 1101 may be transformed from timedomain to the frequency domain 1111. As shown by the solid line in FIG.11B, the first signal contains mainly the baseband component around 6MHz with a bandwidth of approximately 87% (or 5.22 MHz). The spectrum ofthe first signal, however, reveals that a −25 dB second harmoniccomponent and a −35 dB third harmonic component are present in the firstsignal. The first signal also has a −35 dB additive 1/f pink noise.

FIGS. 11C-11E illustrates extraction of the harmonic components withsuitable filters. For example, a 101 tap Finite Impulse Response (FIR)2nd harmonic band pass filter may be applied to the first signal 1101 toextract a filtered 2nd harmonic signal 1102 as shown in FIG. 11D.Additionally, a 3rd harmonic band pass filter (the dash-dotted line inthe bottom right panel), may be applied to the first signal 1101 toextract a filtered 3rd harmonic signal 1103. In some instances, temporalsignals (signals in time domain) may be normalized, and the 2nd and 3rdharmonic signals may be much weaker than the baseband signals. This isbecause tissue generated super-harmonic signals are usually (e.g.,orders of magnitude) lower than the baseband signals. Moreover, higherfrequency signals suffer from larger losses in biological tissues.Without a broad bandwidth sensor such as optical resonators describedherein, and methods and apparatus for compound imaging based on signalsgenerated by the optical resonators harmonic imaging may be extremelydifficult to achieve.

Compounding Algorithms

Exemplary compounding algorithms to combine multiple images based onsignals from non-optical array elements and/or optical resonator arrayelements are described herein. In some instances, n images ofm-dimensions (m-D) are combined (through image compounding) to generatea single m-D image computed as the output (n and m being integers). Whenm is 2, the m-D images are called “images”, while when m is 3, they maybe referred to as “volumes”. Compounding algorithms described may beapplied to both images and volumes. Generally, in some variations,compounding algorithms may produce compounding coefficients (e.g.,factors) that characterize which or how much of each feature(s) (e.g.,pixel intensity) of each separate image (e.g., non-optical image,optical resonator image) may contribute to each compound image. Thecompounding coefficients may be described in a weighting mask that maybe applied to an image to extract the desired features for contributionto a compound image.

In some variations, the compounding algorithm may be or includearithmetic averaging. The idea behind arithmetic averaging for compoundimaging based on signals received from a mixed array is to combine ninput images into one output image with direct pixel-by-pixel arithmeticaveraging of the pixel values:

${I_{f}\lbrack x\rbrack} = {\frac{1}{n}{\sum\limits_{j = 1}^{n}{I_{j}\lbrack x\rbrack}}}$

where, x is the m-D coordinate of a pixel. The n input images mayinclude non-optical array elements and/or optical resonators. In someinstances, the compound images may undergo one or more scalingoperations before being displayed on a fixed dynamic range displaydevice or stored in a database with a predefined dynamic range.

In some variations, the compounding algorithm may be or includegeometric averaging. Similar to the arithmetic averaging methoddescribed above, the geometric averaging method is also a pixel wise(pixel-by-pixel) method performed by:

${I_{f}\lbrack x\rbrack} = \left( {\prod\limits_{j = 1}^{n}{I_{j}\lbrack x\rbrack}} \right)^{1/n}$

In some variations, the compounding algorithm may be or includetransform domain compounding. This is a class of compounding methodsthat relies on transforming the input images into a transform domainthat supports 1-to-1 forward and backward transformations. The 1-to-1transformation may include, for example, a Fourier Transform, a DiscreteWavelet Transform (DWT), a Discrete Cosine Transform (DCT), a Wave AtomTransform, and/or the like. After transformation, a set ofheuristic-based or learned rules may be applied to obtain thecompounding coefficients in the transform domain. Then, the inversetransformation may be performed to convert the compounding coefficientsback to the image domain. An example of this process is shown in FIG. 12. The input images 1202 (non-optical images and/or optical resonatorimages) may undergo a transformation 1204 and coefficients 1206 may begenerated. Coefficient compounding rules 1208 may be applied to thesecoefficients to generate compounding coefficients 1210 in the transformdomain. The compounding coefficients may then be inverse transformed1212 to convert the compounding coefficients to the image domain for usein generating the compound image 1214.

In some variations, transform domain compounding may use transformationthat are suitable for multi-scale analysis of images, such as DWT. Underthe context of DWT, an illustrative example of coefficient compoundingrules includes:

-   -   For the smallest scale among multiple scales, take a minimum        coefficient among coefficients of all images (e.g., non-optical        images, high Q optical image, low Q optical image, and/or the        like). This rule assumes that smallest scale contains mainly        noises and thus should be minimized.    -   For the largest scale among the multiple scales, take an average        of coefficients for all input images. This rule assumes the        largest scale describes the general shape of the object and        should be consistent among the input images.    -   For all other scales (other than the smallest scale and the        largest scale) among the multiple scales, take the maximum of        coefficients among all input images. This rule assumes that all        other scales represent certain details of the target and        different input images may be best in representing one or more        aspects. By taking the maximum, all details may be preserved.

However, if the DWT method is being applied to the method 1000 as shownand described with respect to FIG. 10 , larger weights can be assignedto the smaller scale coefficients of the super-harmonic images and thelarger scale coefficients of the non-optical images.

Additionally or alternatively, a set of coefficient compounding rules(e.g., rules that may be learned, such as through a suitable machinelearning algorithm) may be pre-defined for different ultrasoundfrequencies (e.g., as a lookup table, as a function of ultrasoundfrequency, etc.). For example, a first compounding coefficient (or afirst range of compounding coefficients) may be associated with imagesgenerated using a high ultrasound frequency (or range of high ultrasoundfrequencies), and a second compounding coefficient (or a second range ofcompounding coefficients) may be associated with images generated usinga low ultrasound frequency (or range of low ultrasound frequencies).Generally, in some variations, because higher ultrasound frequenciesattenuate more in far field imaging, compounding coefficients may belower with increasing imaging depth such that images generated using ahigh ultrasound frequency are given less weight in producing thecompounding images.

In some variations, the compounding algorithm may be or include ImageQuality Factor (IQF) based compounding, as shown in FIG. 13 . An imagequality factor (IQF) may be defined as a quantitative measure of imagequality, and may be expressed or otherwise characterized at least inpart by an image quality factor map for an image. There are various IQFsdeveloped for various purposes and applications. For example, each ofand/or any combinations of signal to noise ratio (SNR), entropy, detailresolution, contrast resolution, and penetration depth may be used as anIQF. Different IQFs enhance different aspects of ultrasound images. Insome instances, one or more IQFs 1304 may be extracted from input images1302. The IQFs 1304 are then converted into compounding coefficients1306. The compound image I_(f)(x) 1308 may be calculated by a weightedsum of the input images I_(j)(x),

${I_{f}(x)} = {\left( {\sum\limits_{j = 1}^{n}{{W_{j}\lbrack x\rbrack} \cdot {I_{j}\lbrack x\rbrack}}} \right)/\left( {\sum\limits_{j = 1}^{n}{W_{j}\lbrack x\rbrack}} \right)}$

where x represents the 2D or 3D coordinates, W_(j)[x] is a weightingcoefficient map for the j-th input image. The input images may beoptical resonator image and/or non-optical images depending on thecompound imaging system as shown and described with respect to FIGS. 1-5.

In some variations, the compounding algorithm may be or include localentropy weighted compounding. The local entropy weighted compoundingcombines the input images by assigning weights to each pixel of eachinput image based on the information content in the neighborhood. Thismay be done by computing the entropy of a region surrounding each pixelof each input image. The local entropy of the pixel at the coordinate xin the j-th image may be calculated by:

H _(x,j)=−p_(x,j)·log₂(p _(x,j))

where p_(x,j) is the histogram of the neighborhood of the pixel at thecoordinate x in the j-th image. For this particular pixel, theunnormalized weight may be assigned as:

W _(j) [x]=100^(H) ^(x,j)

Many functions that convert H_(x,j) to a non-negative value may be usedin lieu of this particular example. The compound image may be expressedas:

${I_{f}\lbrack x\rbrack} = {\left( {\sum\limits_{j = 1}^{n}{{W_{j}\lbrack x\rbrack} \cdot {I_{j}\lbrack x\rbrack}}} \right)/\left( {\sum\limits_{j = 1}^{n}{W_{j}\lbrack x\rbrack}} \right)}$

In some variations, the compounding algorithm may be or include fastimage content weighted compounding. As an approximation of localentropy-based weighting, a faster, linear filtering-based algorithm mayalso be used. Instead of computing local entropy of the input images,which could be computationally expensive, W_(j)[x] is computed byapplying a Difference of Gaussian (DoG) filter to the j-th image. Togenerate the compound image the same formula as local entropy weightedcompounding may be used.

In some variations, the compounding algorithm may be or include depthdependent weighted compounding. If the input images have well definedcharacteristics that are depth dependent, a predefined depth-dependentweighting may be useful. The depth dependent weighted compounding may beparticularly helpful when the optical resonator sub-array includes or isoperated as an ultra-sensitive optical resonator (e.g., as shown inFIGS. 3 and 4 ), as some input images can have better quality in theshallower regions and the other images can have better quality in thedeeper regions. Many depth weighting functions may be used, includingbut not limited to linear and gamma functions.

In some variations, the compounding algorithm may be or includesaturation masking. When some input images are prone to signalsaturation (e.g., images produced by high Q optical resonators) or othertype of nonlinearity due to excessive signal amplitude, a saturationmasking step may be introduced to these input images before they are putthrough the compounding methods. Signal saturation may be detected bycomparing the moving average of a beamformed image with a predefinedthreshold. When saturation is detected, the saturated pixels of theinput image under examination may be assigned a zero or close to zeroweight so that its contribution to the compound image will be small andother input image or images, which are not saturated, will dominate.

Although image compounding methods and systems for mixed arrays havebeen described in the context of ultrasound imaging, in some variations,the image compounding methods and systems may be used in applicationsother than ultrasound imaging. For example, in some instances, the imagecompounding methods and systems may be used in computed tomography,magnetic resonance imaging, metrology, signal processing, particlephysics, remote sensing, aerospace applications, and/or the like. Theimage compounding methods disclosed here can also be applied to combineimages generated with different imaging modalities to form a fusedimage. For example, an ultrasound image, a CT image, and an MRI image ofthe same region of a patient can be fused together to show morediagnostic information.

Although, in some variations described above, the tunable opticalresonators are described as operating at a low quality factor (low Q)operation mode or a high quality factor (high Q) operation mode, ingeneral, the tunable optical resonators may be operated in multipleoperation modes (e.g., 3 operation modes, 10 operation modes, 100operation modes). For example, the tunable optical resonators may beoperated at a low Q operation mode to generate a first image having highlinear range, a high Q operation mode to generate a second image havinghigh sensitivity, and a medium quality factor operation mode to generatea third image having a balance between sensitivity and linear range. Theback end of the image compounding system 100 may be configured tocombine the first image, the second image, and the third image togenerate a compound image that is better (e.g., resolution, depth,contrast, quality factor, and/or the like) compared to each of the firstimage, the second image, or the third image.

The foregoing description, for purposes of explanation, used specificnomenclature to provide a thorough understanding of the invention.However, it will be apparent to one skilled in the art that specificdetails are not required in order to practice the invention. Thus, theforegoing descriptions of specific embodiments of the invention arepresented for purposes of illustration and description. They are notintended to be exhaustive or to limit the invention to the precise formsdisclosed; obviously, many modifications and variations are possible inview of the above teachings. The embodiments were chosen and describedin order to explain the principles of the invention and its practicalapplications, they thereby enable others skilled in the art to utilizethe invention and various embodiments with various modifications as aresuited to the particular use contemplated. It is intended that thefollowing claims and their equivalents define the scope of theinvention.

1. A method of imaging comprising: receiving a first signal from one ormore array elements of a first type in a mixed transducer array;receiving a second signal from one or more array elements of a secondtype in the mixed transducer array, wherein at least one of the firsttype and the second type is an optical sensor; generating a first imagefrom the first signal and a second image from the second signal; andcombining the first image and the second image to generate a compoundimage.
 2. The method of claim 1, wherein the first type and the secondtype are optical resonators with different characteristics.
 3. Themethod of claim 2, wherein the first type is a high Q optical resonatorand the second type is a low Q optical resonator.
 4. The method of claim2, wherein the first type is a tunable optical resonator operated as ahigh Q optical resonator and the second type is a tunable opticalresonator operated as a low Q optical resonator.
 5. The method of claim1, wherein the first type is a non-optical transducer and the secondtype is an optical sensor.
 6. The method of claim 5, wherein thenon-optical transducer is a piezoelectric transducer, a single crystalmaterial transducer, a piezoelectric micromachined ultrasound transducer(PMUT), or a capacitive micromachined ultrasonic transducer (CMUT). 7.The method of claim 5, wherein the second type is a broad bandwidthoptical sensor, and wherein the method further comprises receiving athird signal from one or more array elements of a third type, whereinthe third type is an ultra-sensitive optical sensor.
 8. The method ofclaim 7, comprising filtering the first signal, the second signal,and/or the third signal using one or more filters.
 9. The method ofclaim 8, wherein the one or more filters comprise a harmonic band-passfilter.
 10. The method of claim 1, wherein combining the first andsecond images comprises determining an average of the first image andthe second image.
 11. The method of claim 10, wherein combining thefirst and second images comprises determining an arithmetic or geometricaverage of the first image and the second image.
 12. The method of claim10, wherein combining the first and second images comprises determininga weighted average of the first image and the second image.
 13. Themethod of claim 12, further comprising determining one or morecompounding coefficients for the first and second images and combiningthe first and second images based on the one or more compoundingcoefficients.
 14. The method of claim 13, wherein determining one ormore compounding coefficients for the first and second images comprises:transforming the first and second images to first and second transformdomain images using at least one transformation operator; determiningone or more transform domain compounding coefficients for the first andsecond transform domain images; and inverse transforming the one or moretransform domain compounding coefficients to determine the one or morecompounding coefficients for the first and second images.
 15. The methodof claim 14, wherein determining one or more transform domaincompounding coefficients for the first and second transform domainimages comprises applying one or more coefficient compounding rules tothe first and second transform domain images.
 16. The method of claim14, wherein the at least one transformation operator comprises a FourierTransform, a Discrete Wavelet Transform (DWT), a Discrete CosineTransform (DCT), or a Wave Atom Transform.
 17. The method of claim 13,wherein determining one or more compounding coefficients for the firstand second images comprises: determining a first image quality factormap for the first image and a second image quality factor map for thesecond image; and determining a first compounding coefficient for thefirst image based on the first image quality factor map, and a secondcompounding coefficient for the second image based on the second imagequality factor map.
 18. The method of claim 13, wherein determining oneor more compounding coefficients for the first and second imagescomprises determining a local entropy of each pixel in the first imageand in the second image, and determining one or more compoundingcoefficients based on the determined local entropies.
 19. The method ofclaim 13, wherein determining one or more compounding coefficients forthe first and second images comprises applying a linear filter to eachof the first and second images.
 20. The method of claim 19, wherein thelinear filter comprises a Difference of Gaussian filter.
 21. The methodof claim 13, wherein determining one or more compounding coefficientsfor the first and second images comprises determining one or morecompounding coefficients as a function of imaging depth.
 22. The methodof claim 12, wherein determining a weighted average of the first imageand the second image comprises applying a saturation mask that reducesweight of at least a portion of the first image and/or second image thathas exceeded a predetermined saturation threshold.
 23. The method ofclaim 1, wherein the optical sensor is a WGM optical resonator
 24. Themethod of claim 1, wherein the optical sensor is a microbubble opticalresonator, a photonic integrated circuit (PIC) optical resonator, amicrosphere resonator, a microtoroid resonator, a microring resonator, amicrobottle resonator, a microcylinder resonator, or a microdi skoptical resonator.
 25. The method of claim 1, wherein one or morenon-optical transducers in the mixed transducer array transmit acousticsignals at a fundamental frequency f, and the one or more array elementsof the first type, the second type, or both the first type and secondtype, are configured to produce one or more optical responses uponreceiving harmonic or subharmonic acoustic echoes corresponding to thetransmitted acoustic signals, wherein the one or more array elements ofthe second type have a bandwidth ranging from at leastf/Mto Nf, where Mand N are integers greater than
 1. 26. The method of claim 1, whereinthe one or more non-optical transducers transmit acoustic signals at afirst fundamental frequency f₁ and a second fundamental frequency f₂.27. The method of claim 26, wherein one or more array elements of thesecond type are configured to produce one or more optical responses uponreceiving acoustic echoes that correspond to a frequency of one or morelinear combinations nf₁+mf₂, wherein n and m are integers such thatnf₁+mf₂ is a positive number.
 28. The method of claim 1, wherein atleast one of the first image and the second image is a harmonic image.29. The method of claim 28, wherein the harmonic image is a sub-harmonicimage or a super-harmonic image.
 30. An apparatus for imaging a targetcomprising: a mixed transducer array comprising: one or more arrayelements of a first type configured to receive a first signal; one ormore array elements of a second type configured to receive a secondsignal, wherein at least one of the first type and the second type is anoptical sensor; and one or more processors configured to: generate afirst image from the first signal and a second image from the secondsignal; and combine the first image and the second image to generate acompound image.
 31. The apparatus of claim 30, wherein the first typeand the second type are optical resonators with differentcharacteristics.
 32. The apparatus of claim 31, wherein the first typeis a high Q optical resonator and the second type is a low Q opticalresonator.
 33. The apparatus of claim 31, wherein the first type is atunable optical resonator operated as a high Q optical resonator and thesecond type is a tunable optical resonator operated as a low Q opticalresonator.
 34. The apparatus of claim 30, wherein the first type is anon-optical transducer and the second type is an optical sensor.
 35. Theapparatus of claim 34, wherein the non-optical transducer is apiezoelectric transducer, a single crystal material transducer, apiezoelectric micromachined ultrasound transducer (PMUT), or acapacitive micromachined ultrasonic transducer (CMUT).
 36. The apparatusof claim 34, wherein the second type is a broad bandwidth opticalsensor, and wherein the mixed transducer array further comprises one ormore array elements of a third type configured to receive a thirdsignal, wherein the third type is an ultra-sensitive optical sensor. 37.The apparatus of claim 36, wherein the one or more processors areconfigured to filter the first signal, the second signal, and/or thethird signal using one or more filters.
 38. The apparatus of claim 37,wherein the one or more filters comprise a harmonic band-pass filter.39. The apparatus of claim 30, wherein the one or more processors areconfigured to combine the first and second images at least in part bydetermining an average of the first image and the second image.
 40. Theapparatus of claim 39, wherein the one or more processors are configuredto combine the first and second images at least in part by determiningan arithmetic or geometric average of the first image and the secondimage.
 41. The apparatus of claim 39, wherein the one or more processorsare configured to combine the first and second images at least in partby determining a weighted average of the first image and the secondimage.
 42. The apparatus of claim 41, wherein the one or more processorsare configured to determine one or more compounding coefficients for thefirst and second images and combine the first and second images based onthe one or more compounding coefficients.
 43. The apparatus of claim 42,wherein the one or more processors are configured to determine one ormore compounding coefficients for the first and second images at leastin part by: transforming the first and second images to first and secondtransform domain images using at least one transformation operator;determining one or more transform domain compounding coefficients forthe first and second transform domain images; and inverse transformingthe one or more transform domain compounding coefficients to determinethe one or more compounding coefficients for the first and secondimages.
 44. The apparatus of claim 43, wherein the one or moreprocessors are configured to determine one or more transform domaincompounding coefficients for the first and second transform domainimages at least in part by applying one or more coefficient compoundingrules to the first and second transform domain images.
 45. The apparatusof claim 43, wherein the at least one transformation operator comprisesa Fourier Transform, a Discrete Wavelet Transform (DWT), a DiscreteCosine Transform (DCT), or a Wave Atom Transform.
 46. The apparatus ofclaim 42, wherein the one or more processors are configured to determineone or more compounding coefficients for the first and second images atleast in part by: determining a first image quality factor map for thefirst image and a second image quality factor map for the second image;and determining a first compounding coefficient for the first imagebased on the first image quality factor map, and a second compoundingcoefficient for the second image based on the second image qualityfactor map.
 47. The apparatus of claim 42, wherein the one or moreprocessors are configured to determine one or more compoundingcoefficients for the first and second images at least in part bydetermining a local entropy of each pixel in the first image and in thesecond image, and determining one or more compounding coefficients basedon the determined local entropies.
 48. The apparatus of claim 42,wherein the one or more processors are configured to determine one ormore compounding coefficients for the first and second images at leastin part by applying a linear filter to each of the first and secondimages.
 49. The apparatus of claim 48, wherein the linear filtercomprises a Difference of Gaussian filter.
 50. The apparatus of claim42, wherein the one or more processors are configured to determine oneor more compounding coefficients for the first and second images atleast in part by determining one or more compounding coefficients as afunction of imaging depth.
 51. The apparatus of claim 41, wherein theone or more processors are configured to determine a weighted average ofthe first image and the second image at least in part by applying asaturation mask that reduces weight of at least a portion of the firstimage and/or second image that has exceeded a predetermined saturationthreshold.
 52. The apparatus of claim 30, wherein the optical sensor isa WGM optical resonator
 53. The apparatus of claim 30, wherein theoptical sensor is a microbubble optical resonator, a photonic integratedcircuit (PIC) optical resonator, a microsphere resonator, a microtoroidresonator, a microring resonator, a microbottle resonator, amicrocylinder resonator, or a microdi sk optical resonator.
 54. Theapparatus of claim 30, wherein one or more non-optical transducers inthe mixed transducer array transmit acoustic signals at a fundamentalfrequency f, and the one or more array elements of the first type, thesecond type, or both the first type and the second type are configuredto produce one or more optical responses upon receiving harmonic orsubharmonic acoustic echoes corresponding to the transmitted acousticsignal, wherein the one or more array elements of the second type have abandwidth ranging from at leastf/Mto Nf, where M and N are integersgreater than
 1. 55. The apparatus of claim 30, wherein the one or morenon-optical transducers transmit acoustic signals at a first fundamentalfrequency f₁ and a second fundamental frequency f₂.
 56. The apparatus ofclaim 55, wherein one or more array elements of the second type areconfigured to produce one or more optical responses upon receivingacoustic echoes that correspond to a frequency of one or more linearcombinations nf₁+mf₂, wherein n and m are integers such that nf₁+mf₂ isa positive number.
 57. The apparatus of claim 30, wherein at least oneof the first image and the second image is a harmonic image.
 58. Theapparatus of claim 57, wherein the harmonic image is a sub-harmonicimage or a super-harmonic image.